When video is streamed over the Internet and played back through a Web browser or media player, the video is delivered in digital form. Digital video is also used when video is delivered through many broadcast services, satellite services, and cable television services. Real-time videoconferencing often uses digital video, and digital video is used during video capture with most smartphones, Web cameras, and other video capture devices. Digital video is also used for technologies such as virtual reality and augmented reality, whether video is played back in a head-mounted display, mobile device, or other type of device.
Panoramic video is video in which views in multiple directions around a central position are recorded at the same time. The recorded video can include image content in every direction, or at least image content in every direction in a 360-degree sphere around the central position, including at least some image content above the central position and at least some image content underneath the central position. Panoramic video is sometimes called 360-degree video, immersive video, or spherical video. Panoramic video can be captured using an omnidirectional camera or a collection of multiple cameras pointing in different directions. For modern-day applications, panoramic video is processed in digital form during stages of creation, editing, and delivery, as well as stages of reconstruction and rendering for playback.
During playback, a viewer typically can control a view direction relative to the central position, potentially changing which section of the panoramic video is viewed over time. In some systems, a viewer can also zoom in or zoom out, which effectively changes the field of view of the panoramic video. When panoramic video is rendered for display, the section of the panoramic video that is viewed may be projected to a flat (or, at least, non-spherical) image, which is called a screen projection. For a mobile device or computer monitor, a single screen projection may be rendered. For a head-mounted display (or mobile device held in a head-mounted band), the section of the panoramic video that is viewed may be projected to two screen projections, for the left and right eyes, respectively.
Typically, screen projections are rendered by an application. For example, the application may receive a flat projection from a video processing engine, map the flat projection to a sphere, and then render a screen projection from the sphere map. Thus, the processing required to render panoramic video can be complex, making software developers less likely to include panoramic video in their applications. Thus, room for improvement remains in the area of panoramic video rendering.